COST OF LIVING IN THE UK: INSIGHTS VIA AI, DATA VISUALIZATION, AND STORYTELLING

By Amin Sabbagh
Student ID: 18055392
EthOS Reference Number: 64001

Intro | Linear Regression Models | Further Insight | Data Preprocessing Intro | Data Preprocessing Continuous | Data Preprocessing Insights

Imported Libraries

• Pandas: Used to extract data from Excel into dataframes

• Plotly Express: Used to create interactive data visualizations.

import pandas as pd
import plotly.express as px

Further Insight

# Specify the sheet name from which to read data
file_name = "COL.xlsx"
sheet_name = 'Sheet1'

# Read data from Excel file into a DataFrame
df_lines = pd.read_excel(file_name, sheet_name)

normalization for line charts

The min-max approach

# copy the data 
dfn = df_lines.copy() 

#The min-max approach (often called normalization)
for column in dfn:
    if pd.api.types.is_numeric_dtype(dfn[column]):
        dfn[column] = (dfn[column] - dfn[column].min()) / (dfn[column].max() - dfn[column].min())

# View normalized data
display(dfn)

	Unnamed: 0	Date	CPI	Food Inflation	Resturants Inflation	Rental Price	House Price	Motor Fuel Price	Gas inflation
0	0.000000	2016 JAN	0.010989	0.018018	0.314286	0.320755	0.627329	0.256560	0.155860
1	0.010526	2016 FEB	0.010989	0.036036	0.307143	0.320755	0.621118	0.256560	0.155860
2	0.021053	2016 MAR	0.032967	0.009009	0.321429	0.320755	0.664596	0.228863	0.155860
3	0.031579	2016 APR	0.021978	0.022523	0.335714	0.320755	0.633540	0.253644	0.147756
4	0.042105	2016 MAY	0.021978	0.009009	0.342857	0.301887	0.639752	0.263848	0.151496
...	...	...	...	...	...	...	...	...	...
91	0.957895	2023 AUG	0.637363	0.747748	0.814286	0.886792	0.118012	0.123907	0.203865
92	0.968421	2023 SEP	0.637363	0.684685	0.835714	0.905660	0.055901	0.221574	0.203865
93	0.978947	2023 OCT	0.461538	0.590090	0.814286	0.981132	0.062112	0.252187	0.000000
94	0.989474	2023 NOV	0.406593	0.549550	0.771429	1.000000	0.000000	0.208455	0.000000
95	1.000000	2023 DEC	0.406593	0.495495	0.735714	1.000000	0.055901	0.205539	0.000000

96 rows × 9 columns

Insights from Survey on Overall Food Spending

Further insights from a survey conducted across class quintiles reveals widespread food difficulties.

• Key Findings:
  • Across all quintiles, all reported spending less on food shopping and essentials.
  • Most nearing half on spending less, with exception of the 2nd quintile which 52% spends less.
  • This highlights the urgent need for interventions to address food insecurity across different working classes.

# Specify the sheet name from which to read data
file_name = "Bars_Insights.xlsx"
sheet_name = '1'

# Read data from Excel file into a DataFrame
df_food = pd.read_excel(file_name, sheet_name)

# Plot
fig = px.bar(df_food, x='Class', y='Percentage', color='Spending less on food shopping and essentials', barmode='stack', title='Food & Essentials',
            color_discrete_map={'No': px.colors.sequential.BuGn[3], 
                                 'Yes': px.colors.sequential.BuGn[4]},
            hover_data={'Class': True, 'Spending less on food shopping and essentials': True})
fig.show()

The Food Inflation overtime line chart displays two distinct lines: one representing CPI and the other depicting Food Inflation.

• Key Findings:
  • The convergence of these lines at ten distinct points holds considerable significance, providing insights into synchronized fluctuations between overall consumer prices and food-specific inflation.
  • This synchrony suggests a close relationship between general economic conditions and food price dynamics, underlining the interconnectedness of these variables in shaping the UK cost-of-living landscape.
  • Significant temporal patterns are revealed, especially during periods of heightened cost of living.
  • Both CPI and Food Inflation show simultaneous increases, peaking notably in October 2022, coinciding with the peak of the cost-of-living crisis.
  • CPI peaks before Food Inflation, suggesting that general price trends may precede changes in food prices, possibly due to supply chain disruptions or market dynamics.
  • The subsequent peak in Food Inflation in March 2023 highlights a delayed but notable response of food prices to broader economic trends.
  • Monitoring indicators like CPI as leading indicators of future food price movements offers valuable insights for lawmakers and stakeholders addressing the impact of cost-of-living crises on food affordability.

fig = px.line(title='Food Inflation Overtime')

# Add Food trace with green color
fig.add_scatter(x=dfn['Date'], y=dfn['Food Inflation'], mode='lines', name='Food', line=dict(color='green'))

# Add CPI trace with blue color
fig.add_scatter(x=dfn['Date'], y=dfn['CPI'], mode='lines', name='CPI', line=dict(color='blue'))

fig.show()

The Restaurant Inflation overtime line chart presents two distinct lines: one for CPI and the other for Restaurant Inflation.

• Key Findings:
  • While sharing similarities with the Food Inflation overtime chart in terms of temporal patterns, the significance is not as pronounced.
  • The lines intersect only five times, indicating less frequent correlations between general consumer prices and restaurant-specific inflation.
  • However, there is a delayed peak in Restaurant Inflation following the peak in CPI, occurring in February 2023.

fig = px.line(title='Resturant Inflation overtime')

# Add Food trace with green color
fig.add_scatter(x=dfn['Date'], y=dfn['Resturants Inflation'], mode='lines', name='Resturants', line=dict(color='green'))

# Add CPI trace with blue color
fig.add_scatter(x=dfn['Date'], y=dfn['CPI'], mode='lines', name='CPI', line=dict(color='blue'))

fig.show()

Insights from Rent and Mortgage Prices

From a survey conducted across class quintiles, respondents were asked whether rent or mortgages increased after the peak of CPI.

• Key Findings:
  • The results did not indicate widespread difficulties, with less than 25% of each class reporting challenges.
  • Interestingly, the lowest class (1st quintile) appeared to struggle the most, with 22% experiencing difficulties.
  • This disparity suggests that individuals in lower socioeconomic groups may face disproportionate challenges in coping with rising housing costs, underscoring the need for targeted interventions to address affordability issues in this segment of the population.

# Specify the sheet name from which to read data
file_name = "Bars_Insights.xlsx"
sheet_name = '4'

# Read data from Excel file into a DataFrame
df_housing = pd.read_excel(file_name, sheet_name)


# Create the bar chart
fig = px.bar(df_housing, x='Quintile', y='Percentage', title='In Decemeber 2022, My rent or mortgage costs have increased',
             labels={'Percentage': '%', 'Quintile': 'Quintiles'})

fig.update_traces(marker_color = '#66c2a4')


# Update y-axis range
fig.update_yaxes(range=[0, 25]) 

# Show plot
fig.show()

The Rental Price overtime line chart presents two distinct lines: one representing CPI and the other depicting rent price inflation.

• Key Findings:
  • Although there is convergence of these lines at five distinct points, indicating some degree of correlation, there doesn't appear to be a straightforward relationship.
  • Notably, rental prices continue to rise despite CPI decreasing, suggesting some disconnect between overall consumer prices and rent inflation.
  • This discrepancy underscores the complex dynamics influencing rental markets, where factors such as supply and demand, renting policies, and local market conditions can exert significant influence on rental prices independent of broader economic trends.

fig = px.line(title='Rental Price overtime')

# Add Food trace with green color
fig.add_scatter(x=dfn['Date'], y=dfn['Rental Price'], mode='lines', name='Rental Price', line=dict(color='green'))

# Add CPI trace with blue color
fig.add_scatter(x=dfn['Date'], y=dfn['CPI'], mode='lines', name='CPI', line=dict(color='blue'))

fig.show()

The House Price overtime line chart shows two distinct lines for CPI and house price inflation, converging at three points.

• Key Findings:
  • There appears to be little to no relationship between them, despite converging at three points.
  • This suggests that factors influencing house price inflation may extend beyond those captured by CPI, such as housing market dynamics and government policies.

fig = px.line(title='House Prices overtime')

# Add Food trace with green color
fig.add_scatter(x=dfn['Date'], y=dfn['House Price'], mode='lines', name='House Price', line=dict(color='green'))

# Add CPI trace with blue color
fig.add_scatter(x=dfn['Date'], y=dfn['CPI'], mode='lines', name='CPI', line=dict(color='blue'))

fig.show()

Insights from Home Fuel Spending

Based on a survey conducted across class quintiles, respondents were asked about their spending habits on fuel such as gas or electricity in their homes.

• Key Findings:
  • Surprisingly, the majority of respondents across all class quintiles reported spending less on fuel, despite the overall inflation in food prices being higher than gas price inflation.
  • This intriguing trend suggests that individuals may have opted to reduce their usage of heating rather than limiting their overall food consumption.
  • This response underscores the trade-offs individuals make when faced with rising costs, highlighting the prioritization of basic needs such as food over other expenses like heating.
  • Such insights offer valuable perspectives on how households navigate financial constraints and prioritize spending amidst economic challenges.

# Specify the sheet name from which to read data
file_name = "Bars_Insights.xlsx"
sheet_name = '2'

# Read data from Excel file into a DataFrame
df_fuel = pd.read_excel(file_name, sheet_name)



# Plot
fig = px.bar(df_fuel, x='Class', y='Percentage', color='Using less fuel such as gas or electricity in my home\xa0', barmode='stack', title='Fuel',
            color_discrete_map={'No': px.colors.sequential.BuGn[3], 
                                 'Yes': px.colors.sequential.BuGn[4]})
fig.show()

Insights from Gas Inflation Overtime Line Chart

The Gas Inflation overtime line chart reveals interesting dynamics with two distinct lines representing CPI and Gas Inflation.

• Key Findings:
  • The convergence of these lines at eight distinct points holds considerable significance as it provides insights into the synchronized fluctuations between overall consumer prices and gas inflation.
  • This alignment suggests a close relationship between changes in gas prices and fluctuations in the overall cost of living.
  • Even though the number of convergences is less than that of Food (which has ten), the synchronicity highlights the significant impact of gas prices on household budgets.

fig = px.line(title='Gas Inflation overtime')

# Add Food trace with green color
fig.add_scatter(x=dfn['Date'], y=dfn['Gas inflation'], mode='lines', name='Gas inflation', line=dict(color='green'))

# Add CPI trace with blue color
fig.add_scatter(x=dfn['Date'], y=dfn['CPI'], mode='lines', name='CPI', line=dict(color='blue'))
fig.show()

Insights from Motor Fuel Spending

Based on a survey conducted across class quintiles, respondents were queried about their spending habits on motor fuel, such as petrol, particularly in relation to unnecessary trips.

• Key Findings:
  • Surprisingly, nearly half of the respondents across each quintile reported limiting their spending in this area.
  • While this trend is not directly attributed to the inflation of CPI, it suggests that external factors such as the Ukraine war or taxation policies may be influencing consumer behavior.
  • This highlights the complex interplay of various economic and geopolitical factors shaping consumer decisions.

# Specify the sheet name from which to read data
file_name = "Bars_Insights.xlsx"
sheet_name = '3'

# Read data from Excel file into a DataFrame
df_Petrol = pd.read_excel(file_name, sheet_name)

# Plot
fig = px.bar(df_Petrol, x='Class', y='Percentage', color="Cutting back on non-essential journeys in my vehicle\xa0", barmode='stack', title='Petrol usage',
            color_discrete_map={'No': px.colors.sequential.BuGn[3], 
                                 'Yes': px.colors.sequential.BuGn[4]})


fig.show()

The Motor Fuel Inflation overtime line chart depicts two distinct lines representing CPI and Motor Fuel Inflation.

• Key Findings:
  • Despite their convergence at one point, indicating a momentary alignment, there appears to be little consistent connection between the two variables over time.
  • The absence of discernible patterns suggests that motor fuel prices may not consistently impact overall consumer prices.

fig = px.line(title='Motor Fuel Inflation overtime')

# Add Food trace with green color
fig.add_scatter(x=dfn['Date'], y=dfn['Motor Fuel Price'], mode='lines', name='Motor Fuel Price', line=dict(color='green'))

# Add CPI trace with blue color
fig.add_scatter(x=dfn['Date'], y=dfn['CPI'], mode='lines', name='CPI', line=dict(color='blue'))

fig.show()

Closing Remarks

• Primary Impact of Food Inflation: Food inflation emerges as the primary area of impact, but it's essential to recognize that the crisis has broader implications beyond just food prices.

• Allocation of Budget to Essentials: As individuals allocate more of their budget to essentials like food, they may face difficulties in meeting other basic needs, such as heating their homes or saving for emergencies.

• Survey Results: The impact on food costs is evident in the results of the below opinion survey, where working class adults were asked in December 2023 what increased in the cost of living. Food costs impacted the working class the most, followed by gas or electricity costs and then motor fuel costs. Lastly, housing costs seemed to be the least affected.

# Specify the sheet name from which to read data
file_name = "Bars_Insights.xlsx"
sheet_name = '5'

# Read data from Excel file into a DataFrame
df_Intro = pd.read_excel(file_name, sheet_name)

# Define the color scale from darker to lighter blue
color_scale = px.colors.sequential.BuGn

# Creating sidebar plot
fig = px.bar(df_Intro, x="All Adults", y="In December 2023, what increased?", orientation='h', title="In December 2023, what increased in your cost of living?"
             , color="All Adults", color_continuous_scale=color_scale) 

# Customizing hover text to show values in percentage
fig.update_traces(hovertemplate=" %{x}% of Adults")
fig.update_layout(xaxis_title="Percentage (%)", yaxis_title=None)
fig.show()